WORD EMBEDDING

A Study on Using Semantic Word Associations to Predict the Success of a Novel (2021.starsem-1)

Challenge:	The Named Entity Recognition of drugs, medications and chemical entities in Spanish is a new task in the field of NLP .
Approach:	They propose to use SNOMED CT term search engine to classify the entities in Spanish and a neural model for the Named Entity Recognition.
Outcome:	The proposed system achieves 76.29% and 60.34% performance in the Named Entity Recognition and Concept indexing tasks.

Copied to clipboard

Syeda Jannatus Saba, Biddut Sarker Bijoy, Henry Gorelick, Sabir Ismail, Md Saiful Islam, Mohammad Ruhul Amin

Correlations between Word Vector Sets (D19-1)

Challenge:	Existing methods for book success prediction are not effective.
Approach:	They propose to represent a book as a spectrum of concepts based on the association score between its content embedding and a global embeddment for a set of semantically linked word clusters.
Outcome:	The proposed method outperforms the previous methods for book success prediction.

Copied to clipboard

Vitalii Zhelezniak, April Shen, Daniel Busbridge, Aleksandar Savkov, Nils Hammerla

Familiar words but strange voices: Modelling the influence of speech variability on word recognition (2021.eacl-srw)

Challenge:	Similarity measures based on word embeddings are easily competing with deep learning and expert-engineered systems on unsupervised semantic textual similarity tasks.
Approach:	They propose a new approach to measure word embeddings using pooling operations and correlation coefficients instead of pooling . they also propose centered kernel alignment as a natural generalisation of squared cosine similarity for sets of word vectors.
Outcome:	The proposed approach outperforms most recent methods while being much faster and trivial to implement.

Copied to clipboard

Alexandra Mayn, Badr M. Abdullah, Dietrich Klakow

Multimodality

Challenge:	Despite the lack of acoustic-phonetic invariance in speech, listeners can reliably recognize spoken words despite the lack aural-phonemic invariancy.
Approach:	They propose a deep neural model which is trained to retrieve the meaning of a word given its spoken form, a task which resembles that faced by a human listener.
Outcome:	The proposed model is more sensitive to dialectical variation than gender variation and more related to related languages.

Dictionary-based Debiasing of Pre-trained Word Embeddings (2021.eacl-main)

Copied to clipboard

Masahiro Kaneko, Danushka Bollegala

A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking the Privacy-Utility Trade-off (2024.lrec-main)

Challenge:	Existing methods for learning word embeddings using dictionaries do not require access to training resources or knowledge regarding the word embeds used.
Approach:	They propose a method for debiasing pre-trained word embeddings using dictionaries . they learn constraints that must be satisfied by unbiased word embeds from dictionary definitions .
Outcome:	The proposed method removes unfair biases encoded in pre-trained word embeddings while preserving useful semantics.

Copied to clipboard

Stephen Joseph Meisenbacher, Nihildev Nandakumar, Alexandra Klymenko, Florian Matthes

Reliability-aware Dynamic Feature Composition for Name Tagging (P19-1)

Challenge:	Differential Privacy (DP) has been used in NLP for years to address privacy concerns . privacy-enhancing technologies (PETs) are concrete technical solutions that can be incorporated into existing systems.
Approach:	They compare different Differential Privacy algorithms for word-level NLP tasks . they propose concrete steps forward to combat privacy risks in NLP settings .
Outcome:	The proposed methods perform better than the proposed methods on two NLP tasks.

Copied to clipboard

Ying Lin, Liyuan Liu, Heng Ji, Dong Yu, Jiawei Han

YNU-junyi in BioNLP-OST 2019: Using CNN-LSTM Model with Embeddings for SeeDev Binary Event Extraction (D19-57)

Challenge:	Word embeddings are used to encode semantic information, but their quality is not consistent across the vocabulary due to the long-tail distribution of word frequency.
Approach:	They propose a reliability-aware name tagging model that uses word frequency to indicate word quality . they propose to use word frequency-based reliability signals to dynamically select and compose features .
Outcome:	The proposed model outperforms the baseline model on OntoNotes 5.0 and up to 5% gain on cross-genre data sets.

Copied to clipboard

Junyi Li, Xiaobing Zhou, Yuhang Wu, Bin Wang

From Text to Lexicon: Bridging the Gap between Word Embeddings and Lexical Resources (C18-1)

Challenge:	BioNLP 2019 Shared Tasks: binary relation extraction of SeeDev task . Biological information extraction (Bio-IE) is a new field of research .
Approach:	They propose to use convolutional neural networks and long short term memory networks to construct a binary relation extraction model.
Outcome:	The proposed method performed well in the binary relation extraction task.

Copied to clipboard

Ilia Kuznetsov, Iryna Gurevych

Integration of Deep Learning and Traditional Machine Learning for Knowledge Extraction from Biomedical Literature (D19-57)

Challenge:	Distributional word representations are omnipresent in modern NLP.
Approach:	They propose to combine lemmatization and part of speech (POS) typing to improve word embedding performance.
Outcome:	The proposed methods improve word embedding performance on verbs and verbs.

Copied to clipboard

Jihang Mao, Wanli Liu

Enriching Word Embeddings with Domain Knowledge for Readability Assessment (C18-1)

Challenge:	BB system is among the top two systems in five of all six subtasks . knowledge about microbial diversity is crucial for the study of microbiome and bacteria .
Approach:	They present a system that uses word embedding and lexical features to perform entities recognition, normalization and relation extraction.
Outcome:	The proposed system achieves state-of-the-art in five of six subtasks and is among the top two in five.

Copied to clipboard

Zhiwei Jiang, Qing Gu, Yafeng Yin, Daoxu Chen

Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation (N19-1)

Challenge:	Existing word embedding models focus on syntactic or semantic relations of words, while ignoring reading difficulty.
Approach:	They propose a method which learns the word embedding for readability assessment . they extract the knowledge on word-level difficulty from three perspectives to construct a knowledge graph .
Outcome:	The proposed method is effective and potential, the authors show . they use the knowledge-enriched word embedding model on English and Chinese datasets .

Copied to clipboard

Radu Tudor Ionescu, Andrei Butnaru

Norm-Based Curriculum Learning for Neural Machine Translation (2020.acl-main)

Challenge:	a novel word embedding representation for text documents is proposed . the method is based on the Vector of Locally-Aggregated Descriptors used for image representation .
Approach:	They propose a novel representation for text documents based on aggregating word embedding vectors into document embeddables.
Outcome:	The proposed representation improves on the Movie Review data set and is 10% better than the state-of-the-art representation.

Copied to clipboard

Xuebo Liu, Houtim Lai, Derek F. Wong, Lidia S. Chao

ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems (N19-1)

Challenge:	Experimental results show that the proposed method outperforms strong baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x).
Approach:	They propose a norm-based curriculum learning method that measures difficulty, competence and weight of a sentence in a word embedding.
Outcome:	The proposed method outperforms baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x).

Copied to clipboard

Inigo Jauregi Unanue, Ehsan Zare Borzeshi, Nazanin Esmaili, Massimo Piccardi

Robustness-Aware Word Embedding Improves Certified Robustness to Adversarial Word Substitutions (2023.findings-acl)

Challenge:	Existing methods to regularize neural machine translation are limited in low-resource settings.
Approach:	They propose a method that uses regressing word embeddings to regularize neural machine translation.
Outcome:	The proposed system improves on a strong baseline and a state-of-the-art system.

Copied to clipboard

Yibin Wang, Yichen Yang, Di He, Kun He

Robustness in NLP

Challenge:	Embedding interval bound constraint is important for NLP models to be certified robust, but adversarial examples can be crafted by synonym substitutions.
Approach:	They propose a triplet loss to train robustness-aware word embeddings for better certified robustness.
Outcome:	The proposed method outperforms state-of-the-art certified defense baselines and generalizes well to unseen substitutions.

Investigating Capsule Network and Semantic Feature on Hyperplanes for Text Classification (D19-1)

Copied to clipboard

Chunning Du, Haifeng Sun, Jingyu Wang, Qi Qi, Jianxin Liao, Chun Wang, Bing Ma

Information Retrieval

Challenge:	Various neural networks are designed for text classification on the basis of word embedding, but polysemy is a fundamental feature of the natural language, which brings challenges to text classification.
Approach:	They propose to use capsule networks to construct the vectorized representation of semantics and utilize hyperplanes to decompose each capsule to acquire the specific senses.
Outcome:	The proposed model extracts more discriminative semantic features and yields significant performance gain compared to baseline methods.

Cascaded Semantic and Positional Self-Attention Network for Document Classification (2020.findings-emnlp)

Copied to clipboard

Juyong Jiang, Jie Zhang, Kai Zhang

Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning (2020.emnlp-main)

Challenge:	Existing approaches to document classification combine semantic information with positional information (word orders) . document classification is one of the fundamental problems in natural language processing .
Approach:	They propose a new architecture to combine semantic and positional information using a semantic self-attention layer cascaded with Bi-LSTM.
Outcome:	The proposed model can exploit the interaction between semantics and word positions in a more interpretable and adaptive manner while preserving a compact model size and high convergence rate.

Copied to clipboard

Haochen Liu, Wentao Wang, Yiqi Wang, Hui Liu, Zitao Liu, Jiliang Tang

Supervising Unsupervised Open Information Extraction Models (D19-1)

Challenge:	Recent research shows that dialogue systems trained on human conversation data are biased and can produce responses that reflect people’s gender prejudice.
Approach:	They propose a novel adversarial learning framework Debiased-Chat to train dialogue models free from gender bias while keeping their performance.
Outcome:	The proposed framework significantly reduces gender bias in dialogue models while maintaining the response quality.

Copied to clipboard

Arpita Roy, Youngja Park, Taesung Lee, Shimei Pan

Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search (P18-1)

Challenge:	Existing supervised methods that use labeled training data are expensive and difficult to adapt to new domains.
Approach:	They propose a supervised open information extraction framework that leverages unsupervised Open IE systems and labeled data to improve system performance.
Outcome:	The proposed method outperforms existing supervised and unsupervised models by a significant margin.

Copied to clipboard

Jamie Kiros, William Chan, Geoffrey Hinton

Visual Data in NLP

Challenge:	a large-scale lookup operation to ground language via ‘snapshots’ of our physical world accessed through image search is currently used to learn word representations.
Approach:	They propose a large-scale lookup operation to ground language via ‘snapshots’ of our physical world accessed through image search.
Outcome:	The proposed model is based on a large-scale lookup operation to ground language using image search.

Learning Word Embeddings for Low-Resource Languages by PU Learning (N18-1)

Copied to clipboard

Chao Jiang, Hsiang-Fu Yu, Cho-Jui Hsieh, Kai-Wei Chang

Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders (2020.acl-main)

Challenge:	Existing approaches to learn word embedding on a corpus with only a few million tokens are limited to low-resource languages.
Approach:	They propose to use a sparse co-occurrence matrix to factorize the co-existence matrix and validate the proposed approaches in four different languages.
Outcome:	The proposed model is validated in four different languages.

Copied to clipboard

Terra Blevins, Luke Zettlemoyer

Tsetlin Machine Embedding: Representing Words Using Logical Expressions (2024.findings-eacl)

Challenge:	Existing models for Word Sense Disambiguation are not uniformly distributed on rare or unseen senses.
Approach:	They propose a bi-encoder model that embeds the target word with its context and the dictionary definition, or gloss, of each sense.
Outcome:	The proposed model outperforms previous state-of-the-art models on English all-words WSD, with a 31.1% error reduction on less frequent senses over prior work.

Copied to clipboard

Bimal Bhattarai, Ole-Christoffer Granmo, Lei Jiao, Rohan Yadav, Jivitesh Sharma

Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token (2022.findings-emnlp)

Challenge:	Embedding words in vector space is a fundamental first step in state-of-the-art natural language processing.
Approach:	They propose to embed words in vector space using propositional logic instead of dense vectors . they evaluate embeddings on intrinsic and extrinsic benchmarks and visualize word clusters based on their results .
Outcome:	The proposed model outperforms GLoVe on six classification tasks.

Copied to clipboard

Baohao Liao, David Thulke, Sanjika Hewavitharana, Hermann Ney, Christof Monz

Word Embedding Binarization with Semantic Information Preservation (2020.coling-main)

Challenge:	Large-scale pre-trained MLMs can be used to generalize well to a wide range of tasks.
Approach:	They propose to append [MASK]s at a later layer to reduce sequence length for earlier layers.
Outcome:	The proposed method outperforms RoBERTa for 6 out of 8 GLUE tasks on average by 0.4%.

Copied to clipboard

Samarth Navali, Praneet Sherki, Ramesh Inturi, Vanraj Vala

Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation (P19-1)

Challenge:	Word and Sentence embeddings are one of the most common starting points of any NLP task.
Approach:	They propose a way to convert word embedding to binary representation to reduce overall size . they propose different approaches suitable for different downstream tasks based on contextual and semantic information.
Outcome:	The proposed method reduces the size of the embedding while keeping the semantic and relational knowledge intact.

Copied to clipboard

Haipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao

On the Transferability of Adversarial Attacks against Neural Text Classifier (2021.emnlp-main)

Challenge:	Unsupervised bilingual word embedding (UBWE) has helped unsupervised neural machine translation (UNMT) achieve remarkable results in several language pairs.
Approach:	They propose two methods that train UNMT with UBWE agreement . they propose to use UBwe to initialize word embedding in UNMT .
Outcome:	The proposed methods outperform conventional methods on several language pairs.

Copied to clipboard

Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, Kai-Wei Chang

Robustness in NLP

Challenge:	Existing studies show that deep neural networks are vulnerable to adversarial examples . a small perturbation to an input alters the model prediction .
Approach:	They propose a genetic algorithm to find models that can induce adversarial examples to fool models . they propose word replacement rules that can be used for model diagnostics from these examples .
Outcome:	The proposed model can fool almost all existing models, while ignoring the data bias in the training set.

Conversational Word Embedding for Retrieval-Based Dialog System (2020.acl-main)

Copied to clipboard

Wentao Ma, Yiming Cui, Ting Liu, Dong Wang, Shijin Wang, Guoping Hu

Natural Language Interfaces

Challenge:	Existing word embedding methods for retrieval-based dialog systems are based on co-occurrence statistics and train them based upon the same co-existence statistics.
Approach:	They propose a conversational word embedding method which uses the conversation pairs post, reply, and 'reply' they introduce a word alignment model from statistical machine translation and train it on word-level and sentence-level.
Outcome:	The proposed method improves the quality of the selected response on retrieval-based dialog systems.

Contextual String Embeddings for Sequence Labeling (C18-1)

Copied to clipboard

Alan Akbik, Duncan Blythe, Roland Vollgraf

Embedding Words as Distributions with a Bayesian Skip-gram Model (C18-1)

Challenge:	Recent advances in language modeling have made it viable to model language as distributions over characters.
Approach:	They propose to leverage internal states of a trained character language model to produce a new type of word embeddings.
Outcome:	The proposed embeddings outperform the state-of-the-art on four classic sequence labeling tasks.

Copied to clipboard

Arthur Bražinskas, Serhii Havrylov, Ivan Titov

Subword-augmented Embedding for Cloze Reading Comprehension (C18-1)

Challenge:	Rather than assuming that word embeddings are fixed across the entire text collection, we generate them from word-specific prior densities for each word.
Approach:	They propose a method for embedding words as probability densities in a low-dimensional space from a word-specific prior density for each occurrence of a given word.
Outcome:	The proposed method can encode word as a distribution on a range of benchmarks and is comparable to Gaussian embeddings.

Copied to clipboard

Zhuosheng Zhang, Yafang Huang, Hai Zhao

Cross-Lingual Suicidal-Oriented Word Embedding toward Suicide Prevention (2020.findings-emnlp)

Challenge:	Existing models for machine reading comprehension use word and character representations, but character is not the minimal unit.
Approach:	They propose to use subword rather than character for word embedding enhancement . they also empirically explore different augmentation strategies on subword-augmented embedded embedders .
Outcome:	The proposed model outperforms state-of-the-art models on public datasets.

Copied to clipboard

Daeun Lee, Soyoung Park, Jiwon Kang, Daejin Choi, Jinyoung Han

A LSTM Approach with Sub-Word Embeddings for Mongolian Phrase Break Prediction (C18-1)

Challenge:	Existing suicide dictionaries for other languages have been limited to Korean . a model that uses social media data to identify whether a post includes suicidal ideation is useful .
Approach:	They propose a model that uses existing suicide dictionaries for Korean to predict suicidal ideation . they use the existing dictionary for English and Chinese to translate a post into English and then use the separate suicide-oriented embeddings for English.
Outcome:	The proposed model can detect whether a given social media post includes suicidal ideation in Korean . it uses existing suicide dictionaries for other languages to translate the post into English and Chinese, and then embeds the suicide-oriented embeddings for English and China.

Copied to clipboard

Rui Liu, Feilong Bao, Guanglai Gao, Hui Zhang, Yonghe Wang

Analyzing Word Embedding Through Structural Equation Modeling (2020.lrec-1)

Challenge:	Existing word embedding methods for Mongolian PB prediction are expensive and time-consuming.
Approach:	They propose to use Mongolian word embedding to build a robust Mongolian PB prediction model . they encode sub-word units and feed it to LSTM to decode the best corresponding PB label .
Outcome:	The proposed model outperforms traditional model using manual features and achieves 7.49% gain.

Copied to clipboard

Namgi Han, Katsuhiko Hayashi, Yusuke Miyao

Learning Numeral Embedding (2020.findings-emnlp)

Challenge:	Existing studies have shown that word embedding improves accuracy on NLP tasks.
Approach:	They propose a causal diagram based on the evaluation results of word embeddings using partial least squares path modeling.
Outcome:	The proposed model proves that word embedding contributes to solving downstream tasks.

Copied to clipboard

Chengyue Jiang, Zhonglin Nian, Kaihao Guo, Shanbo Chu, Yinggong Zhao, Libin Shen, Kewei Tu

Neural Networks approaches focused on French Spoken Language Understanding: application to the MEDIA Evaluation Task (2020.coling-main)

Challenge:	Existing word embedding methods do not learn numeral embedds well because numerals are limited in number and their appearances in training corpora are highly scarce.
Approach:	They propose two numeral embedding methods that can handle the out-of-vocabulary problem for numerals.
Outcome:	The proposed methods can handle the out-of-vocabulary problem for numerals.

Copied to clipboard

Sahar Ghannay, Christophe Servan, Sophie Rosset

WECA: A WordNet-Encoded Collocation-Attention Network for Homographic Pun Recognition (D18-1)

Challenge:	Recent studies have focused on English language and tasks, but few have explored the complexity of a SLU task.
Approach:	They propose to explore Neural Networks approaches for a French Spoken Language Understanding task.
Outcome:	The proposed approach outperforms classical Neural Network Architectures and achieves state-of-the-art results.

Copied to clipboard

Yufeng Diao, Hongfei Lin, Di Wu, Liang Yang, Kan Xu, Zhihao Yang, Jian Wang, Shaowu Zhang, Bo Xu, Dongyu Zhang

Specializing Word Embeddings (for Parsing) by Information Bottleneck (D19-1)

Challenge:	Homographic puns have a long history in human writing, widely used in written and spoken literature, which intended as jokes.
Approach:	They propose a WordNet-encoded model to settle polysemy of homographic puns and a word weighted model for recognizing them.
Outcome:	The proposed model can distinguish between homographic pun and non-homographic pun texts.

Copied to clipboard

Xiang Lisa Li, Jason Eisner

Handling Out-Of-Vocabulary Problem in Hangeul Word Embeddings (2021.eacl-main)

Challenge:	Pre-trained word embeddings contain rich syntactic and semantic information.
Approach:	They propose a variational information bottleneck method to nonlinearly compress word embeddings . they compress each word embedded to either a discrete tag or a continuous vector .
Outcome:	The proposed method can be extended to other syntactic or semantic tasks.

Copied to clipboard

Ohjoon Kwon, Dohyun Kim, Soo-Ryeon Lee, Junyoung Choi, SangKeun Lee

Unsupervised Joint Training of Bilingual Word Embeddings (P19-1)

Challenge:	Word embedding is considered an essential factor in improving the performance of various Natural Language Processing (NLP) models.
Approach:	They propose a Hangeul word embedding model that infers original word embeds from typos while maintaining high performance.
Outcome:	The proposed model performs well against typos while maintaining high performance.

Copied to clipboard

Benjamin Marie, Atsushi Fujita

Jose Camacho-Collados, Luis Espinosa Anke, Steven Schockaert

Challenge:	Existing methods for unsupervised bilingual word embeddings are limited by the dissimilarity between the word embedded spaces.
Approach:	They propose a method that trains unsupervised bilingual word embeddings jointly on parallel data generated through unsupervised machine translation.
Outcome:	The proposed method outperforms unsupervised mapped bilingual word embeddings in cross-lingual NLP tasks.

Relational Word Embeddings (P19-1)

Copied to clipboard

CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition (N19-1)

Challenge:	Existing approaches to learn word embeddings rely on external knowledge bases . however, they are limited by the amount of available relational knowledge .
Approach:	They propose to encode relational knowledge in a separate word embedding . this is complementary to a standard word embedded from co-occurrence statistics .
Outcome:	The proposed word embedding is complementary to a standard word embed.

Copied to clipboard

Yuying Zhu, Guoxin Wang

Block-wise Word Embedding Compression Revisited: Better Weighting and Structuring (2021.findings-emnlp)

Challenge:	Named entity recognition (NER) in Chinese is essential but difficult because of the lack of natural delimiters.
Approach:	They propose to use a Chinese Named Entity Recognition (NER) model that uses a character-based convolutional neural network and a gated recurrent unit to capture the information from adjacent characters and sentence contexts.
Outcome:	The proposed model outperforms existing models on Weibo, MSRA and Chinese Resume datasets.

Copied to clipboard

Jong-Ryul Lee, Yong-Ju Lee, Yong-Hyuk Moon

Can Domain Adaptation be Handled as Analogies? (L18-1)

Challenge:	Existing methods for word embedding compression are limited . word embeds have a considerable size and need to be compressed to deploy on edge devices .
Approach:	They propose a block-wise low-rank approximation method for word embedding called GroupReduce . they propose 'frequency-inverse document frequency method' and a differentiable method for weighting .
Outcome:	The proposed algorithm more effectively finds word weights than competitors in most cases.

Copied to clipboard

Núria Bel, Joel Pocostales

Information Retrieval

Challenge:	Aspect identification in user generated texts might suffer degradation when changing to other domains than the one used for training.
Approach:	They propose to use offset method to handle domain shifts when there is no available labeled data in a new target domain for an aspect classifier to be retrained.
Outcome:	The proposed method found analogues in the new domain for the initial features but did not deliver the expected results.

HIT - A Hierarchically Fused Deep Attention Network for Robust Code-mixed Language Representation (2021.findings-acl)

Copied to clipboard

Ayan Sengupta, Sourabh Kumar Bhattacharjee, Tanmoy Chakraborty, Md. Shad Akhtar

Augmenting word2vec with latent Dirichlet allocation within a clinical application (N19-1)

Challenge:	linguistics and morphology of resource-short code-mixed texts remain a key challenge in text processing.
Approach:	They propose a hierarchical transformer-based framework that captures the semantic relationship among words and hierarchically learns sentencelevel semantics using a fused attention mechanism.
Outcome:	The proposed framework improves on one European and five Indic languages on four NLP tasks on eleven datasets.

Copied to clipboard

Akshay Budhkar, Frank Rudzicz

Federated Model Decomposition with Private Vocabulary for Text Classification (2022.emnlp-main)

Challenge:	Existing models that combine latent Dirichlet allocation and word embedding for distinguishing between speakers with and without Alzheimer’s disease from transcripts of picture descriptions are not suitable for clinical binary text classification tasks.
Approach:	They propose three models that combine latent Dirichlet allocation and word embedding for distinguishing between speakers with and without Alzheimer’s disease from transcripts of picture descriptions.
Outcome:	The proposed models outperform word2vec and LDA models on a clinical binary text classification task.

Copied to clipboard

Zhuo Zhang, Xiangjing Hu, Lizhen Qu, Qifan Wang, Zenglin Xu

Learning to Perturb Word Embeddings for Out-of-distribution QA (2021.acl-long)

Challenge:	Existing methods to train federated learning (FL) for natural language processing require sensitive data to leave local devices.
Approach:	They propose a fedrated model decomposition method that protects the privacy of vocabularies . they propose an adaptive updating technique to improve the performance of local models .
Outcome:	The proposed method protects the privacy of vocabularies in federated learning tasks . it maintains competitive performance and provides better privacy-preserving capacity compared to status quo methods.

Copied to clipboard

Seanie Lee, Minki Kang, Juho Lee, Sung Ju Hwang

Intra-Correlation Encoding for Chinese Sentence Intention Matching (2020.coling-main)

Challenge:	QA models that are pretraining with unlabeled data can overfit and may not generalize well to unseen data that falls outside the training distribution.
Approach:	They propose a method which perturbs word embedding without changing their semantics.
Outcome:	The proposed method outperforms baseline methods on five target domains on a single source dataset on five different target domain domains.

Copied to clipboard

Xu Zhang, Yifeng Li, Wenpeng Lu, Ping Jian, Guoqiang Zhang

Break it Down into BTS: Basic, Tiniest Subword Units for Korean (2022.emnlp-main)

Challenge:	Existing methods to improve sentence intention matching for Chinese text are limited due to the particularity of the text.
Approach:	They propose a method that combines character-granularity and word-granulularity features to perform sentence intention matching.
Outcome:	The proposed method can capture sentence feature information from multiple perspectives and correlation information between different levels of sentences.

Copied to clipboard

Nayeon Kim, Jun-Hyung Park, Joon-Young Choi, Eojin Jeon, Youjin Kang, SangKeun Lee

Bilingual Lexicon Induction through Unsupervised Machine Translation (P19-1)

Challenge:	Existing word embeddings for Korean use the internal structure of words with subword information to improve the quality of word representations.
Approach:	They introduce Basic, Tiniest Subword (BTS) units for Korean language that are inspired by Hangeul, the Korean writing system.
Outcome:	The proposed framework outperforms the state-of-the-art Korean word embedding by 11.8% on all intrinsic and extrinsic tasks.

Copied to clipboard

Mikel Artetxe, Gorka Labaka, Eneko Agirre

Offensive Video Detection: Dataset and Baseline Results (2020.lrec-1)

Challenge:	Existing methods for bilingual lexicon induction use nearest neighbor or related retrieval methods to induce word translation pairs.
Approach:	They propose a method that aligns word embeddings in two languages and uses them to build a phrase-table and a language model to extract the bilingual lexicon.
Outcome:	The proposed method improves accuracy 6 points over nearest neighbor and 4 points over CSLS retrieval on the same cross-lingual embeddings.

Copied to clipboard

Cleber Alcântara, Viviane Moreira, Diego Feijo

Information Retrieval

Challenge:	a large number of social media platforms discourage users from publishing offensive content . however, there is no method to detect offensive content on these platforms due to the high volume of publications.
Approach:	They propose to use text-based machine learning to detect offensive content on different platforms . they use word embedding with Deep Learning classifiers to perform best results .
Outcome:	The proposed methods outperform Classic and Deep Learning classifiers in Portuguese and CNN architectures in other features.

Examining Gender Bias in Languages with Grammatical Gender (D19-1)

Copied to clipboard

Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, Kai-Wei Chang

Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation (2020.acl-main)

Challenge:	Existing studies on gender bias in word embeddings focus on English . however, these studies cannot be extended to languages with morphological agreement on gender .
Approach:	They propose new metrics to evaluate gender bias in word embeddings of English and Spanish . they extend existing approaches to mitigate gender bias while preserving original embeddables .
Outcome:	The proposed methods reduce gender bias while preserving the original embeddings.

Copied to clipboard

Shun-Po Chuang, Tzu-Wei Sung, Alexander H. Liu, Hung-yi Lee

BD-SHS: A Benchmark Dataset for Learning to Detect Online Bangla Hate Speech in Different Social Contexts (2022.lrec-1)

Challenge:	Existing studies show that multitask learning improves speech translation performance by utilizing word embedding as the intermediate.
Approach:	They propose to use word embedding as an intermediate to improve multitask ST models by utilizing word embeds as input.
Outcome:	The proposed model outperforms existing models with sufficient training data but is still lacking in the low-resource scenario.

Copied to clipboard

Nauros Romim, Mosahed Ahmed, Md Saiful Islam, Arnab Sen Sharma, Hriteshwar Talukder, Mohammad Ruhul Amin

Ethical NLP

Challenge:	Social media platforms and online streaming services have spawned a new breed of Hate Speech (HS) due to the massive amount of user-generated content, modern machine learning techniques are feasible and cost-effective to tackle this problem.
Approach:	They propose to use a large manually labeled Bangla HS dataset to train generalizable models.
Outcome:	The proposed dataset includes more than 50,200 offensive comments crawled from online social networking sites and is at least 60% larger than existing Bangla HS datasets.

Neural Language Modeling for Named Entity Recognition (2020.coling-main)

Copied to clipboard

Zhihong Lei, Weiyue Wang, Christian Dugast, Hermann Ney

Contrastive Learning-based Sentence Encoders Implicitly Weight Informative Words (2023.findings-emnlp)

Challenge:	Experimental results show that named entity recognition systems are faster and more flexible for the size of the corpus.
Approach:	They propose to use a neural language model as an alternative to the conditional random field layer for named entity recognition.
Outcome:	The proposed system has a significant speed advantage with a marginal performance degradation.

Copied to clipboard

Hiroto Kurita, Goro Kobayashi, Sho Yokoi, Kentaro Inui

Improving word mover’s distance by leveraging self-attention matrix (2023.findings-emnlp)

Challenge:	Embedding a sentence into a point in a highdimensional continuous space plays a foundational role in the natural language processing.
Approach:	They propose to use contrastive loss to fine-tune sentences by inverse word frequency . they also show that more informative words receive greater weight than less informative ones .
Outcome:	The proposed method improves the performance of sentence embeddings by weighing them based on information-theoretic quantities.

Copied to clipboard

Hiroaki Yamagiwa, Sho Yokoi, Hidetoshi Shimodaira

Assessing the Reliability of Word Embedding Gender Bias Measures (2021.emnlp-main)

Challenge:	Word mover's distance (WMD) computes semantic similarity via the optimal alignment between word embeddings, but does not consider word order, making it challenging to distinguish sentences with significant overlaps of similar words.
Approach:	They propose to incorporate the sentence structure represented by BERT’s self-attention matrix (SAM) to improve WMD and its variants in paraphrase identification.
Outcome:	The proposed method improves WMD and its variants in paraphrase identification with near-equivalent performance in semantic textual similarity.

Copied to clipboard

Yupei Du, Qixiang Fang, Dong Nguyen

Knowledge-augmented Graph Neural Networks with Concept-aware Attention for Adverse Drug Event Detection (2024.lrec-main)

Challenge:	Various measures have been proposed to quantify human-like social biases in word embeddings, but they can suffer from measurement error.
Approach:	They propose to assess the reliability of word embedding gender bias measures by examining their reliability across different choices of random seeds, scoring rules and words.
Outcome:	The proposed measures can suffer from measurement error, and the results inform better design of word embedding gender bias measures.

Copied to clipboard

Ya Gao, Shaoxiong Ji, Pekka Marttinen

Quantum Recurrent Architectures for Text Classification (2024.emnlp-main)

Challenge:	Recent studies have used word embedding and deep learning to automate ADE detection from text, but they did not incorporate explicit medical knowledge about drugs and adverse reactions or the corresponding feature learning.
Approach:	They propose to integrate medical knowledge into ADE detection from text . they use contextualized embeddings from pretrained language models and convolutional graph neural networks to learn features differently for different types of nodes in the graph.
Outcome:	The proposed model outperforms existing models on four public datasets and shows that it is based on medical knowledge and embeddings from pretrained language models and neural networks.

Copied to clipboard

Wenduan Xu, Stephen Clark, Douglas Brown, Gabriel Matos, Konstantinos Meichanetzidis